Various manufacturing processes and other industrial processes require inspection of manufactured and processed parts to identify and/or classify defects.
Automated inspection methods using machine vision techniques are commonly used to perform such inspections. Due to the vast number of possible surface qualities and textures, machine vision applications that are useful for inspecting certain surface types may not be well suited for inspecting certain different surface types. Variations in image feature contrast and reflectivity, for example, can hide critical defects in a machine vision image.
Optical fiber manufacturing operations, for example, use various machine vision inspection techniques for quality assurance or in-process inspections of fiber ends. In fiber-optic cable production, once a fiber has been cleaved, it is polished to ensure its end face is smooth and clean. During the polishing process, the fiber is typically placed against an abrasive disc that polishes the fiber end surface. This process can cause defects, which if undetected can block the proper transmission of light through a fiber. Most fiber end defects can be categorized into one of four categories: spots, scratches, cladding boundary defects, and cracks. Lightwave signals that are transmitted through optical fibers are degraded when photons that make up the light wave encounter such defects and are deflected. While certain defects are considered acceptable and will not interfere with transmission of lightwave signals, others—especially those near the core of the fiber—may be critical. Fiber end defects must be identified and classified during the manufacturing process so that the fibers can be cleaned, repolished or discarded entirely, depending on the size and location of the defects.
Optical fiber manufacturers have traditionally relied on manual inspection to detect end surface defects that remain after polishing. One typically performs such manual inspections by inserting a cable into a fiber microscope, which is also called a fiberscope, and analyzing a magnified view of the fiber end connector (which includes the fiber end and ferrule). Although manual inspection methods are expensive, the alternative, i.e., machine vision inspection of optical fiber end surfaces, has proven difficult to implement. Optical fiber end surface inspection has presented many obstacles to the application of automated inspection techniques using machine vision systems. While standard machine vision methods are adequate for many industrial machine vision applications, highly specialized operations, like inspecting polished fiber ends, often require specialized vision methods. A machine vision method implemented in software and/or hardware is typically called a vision “tool.”
Scratch defects on fiber end surfaces appear as straight or curved lines in an acquired image. Machine vision systems used for inspecting fiber ends detect these lines to locate defects. The lines often have very low contrast and are speckled with noise and texture artifacts and therefore do not always appear continuous to common edge detection techniques that use local metrics or small kernel sizes. Such contours that appear discontinuous in an image are particularly difficult for machine vision systems to identify. These straight or curved contours are the image features that are to be identified and located when inspecting fiber ends for scratches.
One common technique well known in the art of computer vision is called binarization, where pixels in an image are classified into two groups. One group typically consists of pixels with a grey level value above a threshold and the other consists of pixels with a grey level below the same threshold. Once pixels are labeled as belonging to one of two groups, a technique called connectivity analysis can be performed. Connectivity analysis considers pixels that touch each other either at the sides, top or bottom or diagonally at the corners to be connected. Groups of connected pixels are commonly referred to as blobs. This technique is commonly referred to as blob analysis. Once blobs are extracted from an image, they can be analyzed by calculating various metrics, including but not limited to area, first and second order moments, location, orientation and perimeter. The metrics could then be used to identify scratch-like features in an image.
The advantages of binary techniques such as blob are simplicity and high speed. However, the major shortcoming of blob and other binary techniques is that a gray level threshold must be calculated. In actual applications, a separate processing step must be run to determine a binarization threshold. This is not always easy to do given that there may be many confusing features in the image. In many cases, especially in fiber and inspection, a single threshold may not exist due to low frequency intensity gradients across the image caused by the backlighting effects and the existence of the fiber core. Another shortcoming of binary techniques is that they only provide whole pixel accuracy. Also, scratches can have low contrast thus making it very difficult to find a usable threshold. When the texture and noise in the image has intensity values that vary from the mean of the background by as much as the contrast of the scratch itself, a binarization of the image will include many extraneous pixels in the same group as the scratches. This extraneous data must then be factored out when recognizing and localizing scratches.
Another common method of image analysis used in industry is referred to as template matching, in which a template is compared with like-sized subsets of the image over a range of positions. The position having the greatest match with the template is determined to be the position of the object in the image. A numeric representation of the degree of match can be generated and used for inspection purposes. One disadvantage of template matching methods is that they require a training step wherein a model image or template of objects to be located is generated and stored. For the inspection of fiber end surfaces, it is very difficult to generate models of defects such as scratches due to their variability. One would have to create models of potential scratches at many possible angles, lengths and curvatures. All these models would then have to be searched for in an image. This type of exhaustive technique is impractical since it would not run in a reasonable amount of time. There are many different types of template matching techniques used in computer vision such as matching with binary images, matching with grey scale images and normalized correlation which can handle differences in the mean pixel grey level values between model and image. All of these variations on template matching suffer from the same shortcoming as mentioned above.
A method of geometric pattern matching was developed to overcome the limitations of image-intensity-based template matching methods by representing objects as geometric shapes independent of shading and independent of a discrete grid. The geometric pattern matching method can tolerate and measure wide variations in orientation and size of objects in an image, can be performed with high speed and high accuracy, and is able to tolerate severe shading variations. An exemplary geometric pattern-matching tool called PatMax® is available from Cognex Corporation of Natick, Mass.
The geometric pattern matching technology used in PatMax includes methods for feature detection, training, affine searching, pose refinement, and inspection. Feature detection produces a geometric description of object boundaries as an ordered list of boundary points that lie along contours separating dissimilar regions in the image. Unlike traditional edge detection, which simply marks certain pixels as edges, each boundary point specifies positions and orientation as real-valued coordinates.
The geometric descriptions of objects produced by feature detection can be translated, rotated, scaled, stretched and otherwise transformed with no loss in fidelity. PatMax uses a six-degree-of-freedom affine transform which includes two degrees of freedom of translation and one each of rotation, scale, aspect ratio and shear. Each non-translation degree of freedom is specified as either a fixed value or a range of values to be searched. A fixed value for a degree of freedom allows a transformed version of the trained pattern to be found with no time penalty. For example, a single square pattern can be trained and used to find rectangular objects at a variety of known sizes and shapes by setting appropriate fixed values for the scale degrees of freedom.
In order to recognize an object in an image, analysis tools, such as PatMax, must first be trained by acquiring and storing an image of a representative object or model. The model can be developed as an image of an actual object or can be synthetically constructed using CAD tools. During runtime operation, the machine vision system acquires an image that may include the object of interest at any scale or orientation relative to the viewing device. The analysis tools will recognize the object in the image and compute the degree of rotation, scale and translation of the object in the image relative to the trained model. This computation is sometimes referred to as an alignment of the object. Alignment may also allow other degrees of freedom.
While PatMax has some flexibility in performing a rotation and scale invariant search, it is not optimal for finding scratches on fiber ends. One reason is that scratches that appear as lines are very self-similar along the line itself. Geometric pattern matching techniques will have difficulty localizing line models along lines in an image. Scratches may be of any length and finding lines of varying lengths with a single line model of a fixed length that can be scaled has problems. The tool will find a good match in the dimension perpendicular to the line, but along the line there can be many potential matches, all with an equal score. The many good matches can make the tool take a long time to executed and when the line model is allowed to change scale, the match calculation can become numerically unstable. PatMax is also susceptible to the problems with lower-level edge detectors as described next. PatMax also requires moderately sharp edge features with reasonable contrast.
Also, both template matching and geometric pattern matching techniques assume that the object to be found will have a consistent appearance, and this is usually not the case for scratches in a fiber end face. For example, the scratches can have a range of curvatures, and some may even have sharp turns. Thus, there is typically no affine transformation that could match a given curved scratch pattern to an actual curved scratch, since the shape may be different.
Scratches can be particularly difficult to detect because they usually have low contrast with respect to a fiber end surface. Various lighting methods can increase the contrast of scratches and cracks during image acquisition so that they can be more easily detected. For example, viewing a surface directly while applying side lighting, viewing a surface at an angle with lighting from the opposite angle, or transmitting light through the fiber core can make scratches and cracks more visible. These types of lighting and viewing angle changes can enhance the quality of the scratch contrast but still will not be enough to make simple vision techniques work.
Another method of detecting scratches is to project the image of the fiber end surface to form a one-dimensional (1D) projection image. This type of projection technique is well known in the art and involves summing the pixels along rows of an image such that the resulting image is a one-dimensional image that has as many elements as rows in the source image and each element contains the sum of all the pixels in a row. Lines that are lighter than the background and that are horizontal will appear as peaks in the 1D projection image. The location of the line is then determined from the row in which the peak occurred. This is generalizable to all angles where the image is rotated into position such that the candidate angle aligns to the horizontal rows. The shortcomings of this type of projection technique are that the length of lines cannot be determined and that only straight lines can be found. Projections are also very susceptible to confusion by bright spots. A bright spot appears on a projection exactly like a line with lower contrast.
Many machine vision methods require extracting what are called primitive or low-level features and then building more complex objects or features from those simple ones. The most basic low-level feature is a single pixel that has a grey value. A more complex feature would be an edge feature that results from applying a Sobel or Roberts operator that is well known in the art. These are small kernel operators (3×3 and 2×2) that return the direction and magnitude of the intensity gradient in the image at the location of the center of where the operator is applied. These operators are known in the art as edge detectors. While the previous techniques discussed (blob and template matching) analyzed the pixels of an image directly, the following techniques that have been used for scratch detection rely on processing the results of edge detectors that are called edge features.
Commonly used edge detectors have several shortcomings when used to detect scratches on fiber end surfaces. Scratches can be irregular at a small scale (1 or 2 pixels) but still appear uniform and straight at a larger scale. Scratches can also be faint and have low contrast. Since the support of the edge operator is no larger than 3×3, only sharp higher contrast edge features will be detected. Also, the small-scale irregularities of the scratches will generate many false edges and miss many valid edges such that the subsequent processing steps, such as chaining, have too many features to consider, thus making such processing run slowly and produce erroneous results.
Often the problem of detecting and tracing scratches becomes a problem of grouping features such as edge features to form higher-level contours. One well known method of grouping low-level features into contours or lines that can be called scratches is to perform a Hough transform on the image space.
The Hough transform aids in the detection of lines or arcs of a particular shape. Each arc or line has some free parameters that, when specified precisely, define the arc or line. The free parameters constitute the transform domain or the parameter space of the Hough transform. Pixels or other features in an image are mapped into the Hough parameter space, which is quantized into discrete bins called accumulator cells (a multidimensional array). Peaks in the accumulator array indicate that a line or an arc may exist at corresponding points in an image. See Robert M. Haralick and Linda G. Shapiro, Computer and Robot Vision, Volume II, p. 184 (1993).
When applied to identify straight lines, an implementation of the Hough transform, called “line Hough,” is used; typically, edge features of an image are used as the immediate inputs to line Hough. The line Hough is limited to identifying straight lines and, by design, detects lines that are very sparse. For example, two short segments on opposite sides of an image that happen to be co-linear will be reported as a line even if they do not correspond to the same continuous line segment. The line Hough has additional limitations including the generation of large numbers of candidates that must each be separately verified.
A more general Hough transform is used to find curves of specific shapes and sizes. Arbitrary curves or contours, such as some fiber end scratches, however, cannot be found by any implementation of the Hough transform. Furthermore, the quantization of Hough space into accumulator cells can introduce significant error in the reported location of candidate lines or curves. Hough transforms also typically rely on lower level edge detection such as Sobel or Robert transforms and therefore suffer all of the limitations of those lower level operations.
Unlike Hough transform methods, chaining methods operate locally and do not connect unrelated pixels that may happen to fall on a predefined contour. However, the local scope of chaining methods can prevent them from bridging small gaps to connect edge segments that do belong to the same contour. Such small gaps occur quite frequently in images and are especially problematic in low contrast defects such as scratches in optical fiber end surfaces.
U.S. Pat. No. 5,995,212 to Dar et al. (hereinafter referred to as “the Dar patent”), which is hereby incorporated by reference in its entirety, describes a method of inspecting an optical fiber for scratches using a window technique which implements the Hough transform. The method according to the Dar patent disadvantageously requires extensive pre-processing to isolate regions of suspected scratches in which to apply the Hough transform. More significantly, the method according to the Dar patent suffers the same disadvantages of the general Hough transform in that it connects widely separated pixels that may not be part of a common contour.
Another approach to identifying contours in an image is chaining of edge features. U.S. Pat. No. 5,179,419 to Palmquist et al. (hereinafter referred to as “the Palmquist patent”), which is hereby incorporated by reference in its entirety, describes a method for detecting, classifying and quantifying defects including scratches. The method taught by the Palmquist patent breaks edge features in an image into segments to distinguish overlapping defects. Connected segments are decomposed and labeled to group pixels based on connectedness. Segments are connected to follow curves and break at sharp corners. Small segments caused by noise are discarded. The method of chaining edge segments taught by the Palmquist patent uses well-known filters such as Sobel filters. The chaining steps performed by Palmquist are much like base chaining functions performed by several image analysis tools, including PatMax.
The method disclosed by the Palmquist patent disadvantageously requires the acquisition of an optimal focus image and auxillary images behind and in front of the optimal focal positions. Since the method in the Palmquist patent uses simple Sobel edge detection, it suffers from all the shortcomings associated with this type of feature extraction technique as described previously.
U.S. Pat. No. 6,167,150 to Michael et al. (hereinafter referred to as “the Michael patent”), which is hereby incorporated by reference in its entirety, discloses a method of detecting edges in an image that represent scratches in disk drive heads. The method disclosed in the Michael patent detects edges in the image and determines if edges are connected using the magnitude and orientation of the edges. After eliminating edges based on magnitude, the orientation of neighboring edges are compared and are determined to form connected edges when the angle of the neighboring edges are within a pre-determined range of each other.
The method disclosed in the Michael patent works well to detect very thin scratches in the order of one pixel wide because it is designed to locate edges. Wider scratches, in the order of five to ten pixels wide, however, are recognized as two separate entities because each edge of such wider scratches is separately detected. This redundant reporting of found features requires additional processing, for example, to associate two edges with a common scratch feature. In addition, fiber scratches that are irregular, low in contrast, or formed by a series of dots that are not necessarily connected, do not form continuous connected edges and will not be detected by this method. In fact, edge detectors may not detect a string of dots at all.